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Abstract 


Decision scientists have identified various plausible sources of ideological polarization over climate change, gun 
violence, national security, and like issues that turn on empirical evidence. This paper describes a study of three of 
them: the predominance of heuristic-driven information processing by members of the public; ideologically motivated 
reasoning; and the cognitive-style correlates of political conservativism. The study generated both observational and 
experimental data inconsistent with the hypothesis that political conservatism is distinctively associated with either un- 
reflective thinking or motivated reasoning. Conservatives did no better or worse than liberals on the Cognitive Reflection 
Test (Frederick, 2005), an objective measure of information-processing dispositions associated with cognitive biases. In 
addition, the study found that ideologically motivated reasoning is not a consequence of over-reliance on heuristic or 
intuitive forms of reasoning generally. On the contrary, subjects who scored highest in cognitive reflection were the 
most likely to display ideologically motivated cognition. These findings corroborated an alternative hypothesis, which 
identifies ideologically motivated cognition as a form of information processing that promotes individuals’ interests in 
forming and maintaining beliefs that signify their loyalty to important affinity groups. The paper discusses the practical 
significance of these findings, including the need to develop science communication strategies that shield policy-relevant 


facts from the influences that turn them into divisive symbols of political identity. 
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1 Introduction 


Ideological polarization is a conspicuous but peculiar fea- 
ture of American democracy. No one is surprised when 
conservatives and liberals fight over tradeoffs between 
wealth and equality or between “law and order” and civil 
liberties. Differences in the value attached to such goods 
define those political outlooks. 

Much more perplexing, however, are the ubiquity and 
ferocity of ideological conflicts over facts that turn on 
empirical evidence. Democrats (by and large) fervently 
believe that human activity is responsible for global 
warming, Republicans (by and large) that it is not (Pew 
Research Center, 2012). Conservatives are confident that 
the wastes generated by nuclear power plants can be 
safely disposed of by deep geologic isolation; liberals 
dispute that (Jenkins-Smith, Silva, Nowlin & deLozier, 
2011). People who value equality and community be- 
lieve that vaccinating schoolgirls against the human pa- 
pilomavirus is essential to protecting women’s health— 
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and that permitting private citizens to carry concealed 
hand guns increases crime. Those who value hierarchy 
and individualism, in contrast, reply that universal HPV 
vaccination will undermine young girls’ health by lulling 
them into unprotected sex, and that gun control increases 
crime by making it harder for law-abiding citizens to pro- 
tect themselves (Kahan, Braman, Cohen, Gastil & Slovic, 
2010; Kahan, 2010). 

Political polarization on empirical issues like these oc- 
curs not only despite the lack of any logical connection 
between the contending beliefs and the opposing values 
of those who espouse them. It also persists despite appar- 
ent scientific consensus on the answers to many of these 
disputed questions (Lewandowsky, Gignac, & Vaughan, 
2012; Kahan, Jenkins-Smith & Braman, 2011). 

Decision science suggests a range of potential explana- 
tions for why dueling assertions of fact occupy frontline 
positions in U.S. culture wars. It is well established that 
members of the public rely on heuristics or mental short- 
cuts that can generate systematic biases in their percep- 
tions of risk and similar facts. They also tend to seek out 
and assess evidence in biased patterns that reinforce the 
positions that they, or those who share their ideological 
predispositions, already hold. Some psychologists main- 
tain, too, that these effects are intensified by dogmatism, 
aversion to complexity, and like traits that correlate with 
political conservativism and that make politically conser- 
vative individuals distinctively resistant to revising their 
beliefs based on empirical evidence. 
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All of these positions are firmly rooted in empirical 
study, but relatively little testing has been done on how 
they relate to one another. More than one plausible ac- 
count exists of how the various dynamics that connect 
ideology and factual perceptions interact. Empirical test- 
ing of these competing surmises is necessary, not only 
to advance understanding of ideological polarization over 
policy-relevant facts but also to guide practical action 
aimed at mitigating it. 

This paper reports the results of a study designed to 
contribute to the necessary testing process. Using both 
observational and experiment methods, the study exam- 
ined alternative hypotheses relating to how three psy- 
chological dynamics—dual process reasoning, the cogni- 
tive style associated with conservativism, and motivated 
cognition—combine to generate polarization over risks 
and other policy consequential facts. 


2 Empirical and theoretical back- 
ground 


Scholars have identified three important influences that 
contribute to public controversy over factual questions 
that turn on scientific and related forms of empirical ev- 
idence. This part briefly describes these influences and 
identifies alternative conjectures about how they interact 
with one another. 

The first dynamic associated with public controversy 
over policy-relevant facts is heuristic-driven information 
processing. Long prominent in the study of psychology, 
dual-process theories posit two modes of information- 
processing: a “fast, associative” one “based on low-effort 
heuristics”, and a “slow, rule based” one that relies on 
“high-effort systematic reasoning” (Chaiken & Trope, 
1999, p. ix). Many scholars attribute controversy over so- 
cietal risks to the disposition of members of the public to 
over-rely on the heuristic-driven, “System 1” (Stanovich 
& West, 2000; Kahneman, 2003) reasoning style. The 
centrality of visceral, emotion-guided modes of percep- 
tion can cause laypeople to overestimate the incidence 
and harm associated with more sensational risks—such 
as terrorist acts and gun accidents—relative to more re- 
mote, less gripping hazards such as climate change and 
swimming pools. Expert opinion does not reliably cor- 
rect these distortions because members of the public too 
often lack the time or ability to engage in the more ef- 
fortful, more dispassionate “System 2” style of reason- 
ing suited to understanding the technical evidence that 
experts use to assess risks (Loewenstein, Weber, Hsee 
& Welch, 2001; Sunstein, 2003, 2006a, 2007; Weber, 
2006). 

The second dynamic that generates conflict over risk 
issues is motivated reasoning. Motivated reasoning refers 
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to the tendency of people to conform assessments of 
information to some goal or end extrinsic to accuracy 
(Kunda, 1990; Balcetis & Dunning, 2008; Dunning, 
1999; Ditto, Pizarro & Tannenbaum, 2009). 

The goal of protecting one’s identity or standing in 
an affinity group that shares fundamental values can 
generate motivated cognition relating to policy-relevant 
facts (Cohen, 2003; Sherman & Cohen, 2006). Even 
among modestly partisan individuals, shared ideologi- 
cal or cultural commitments are likely to be intertwined 
with membership in communities of one sort or an- 
other that furnish those individuals with important forms 
of support—emotional and psychic as well as material 
(Green, Palmquist & Schikler, 2002). If a proposition 
about some policy-relevant fact comes to be commonly 
associated with membership in such a group, the prospect 
that one might form a contrary position can threaten one’s 
standing within it. Thus, as a form of “identity self- 
defense,” individuals are unconsciously motivated to re- 
sist empirical assertions—that the death penalty deters or 
does not deter murder, for example (Lord, Ross & Lep- 
per, 1979), or that gun control reduces or does or does not 
reduce crime (Taber & Lodge, 2006)—if those assertions 
run contrary to the dominant belief within their groups 
(Cohen, Bastardi, Sherman, Hsu, McGoey, & Ross, 2007; 
Liu & Ditto, 2013; Munro & Ditto, 1997). 

The third influence linked to such controversies is the 
association between ideological or cultural values and 
cognitive-reasoning styles. Refining the 1950s “authori- 
tarian personality” theory of Adorno and his collaborators 
(1950), a substantial body of empirical study generated in 
the last decade has revived interest in, and compelled re- 
spectful scholarly engagement with, the hypothesis that 
right-wing ideology is a manifestation of settled intellec- 
tual traits such as dogmatism, aversion to complexity, and 
a craving for certainty or “closure” in argumentation. The 
cognitive style that comprises these dispositions, it is sur- 
mised, generates reflexive closed-mindedness toward em- 
pirical evidence hostile to the factual premises of policies 
that reflect ideologically conservative values or policy 
preferences (Jost, Glaser, Kruglanski & Sulloway, 2003; 
Jost, Nosek & Gosling, 2008; Feygina, Jost & Goldsmith, 
2010). 

While all of these positions are amply supported by 
empirical evidence, how they interact to generate polar- 
ization over empirical evidence relevant to societal risks 
and like facts has not been systematically explored (Jost, 
Hennes & Lavine, in press). Multiple alternative relation- 
ships are all plausible. 

It would not be surprising, for example, to discover 
that the impact of heuristic-driven reasoning is the most 
decisive one. The predominance of System 1 reason- 
ing among members of the general public, on this view, 
would account for the failure of democratic institutions to 
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reliably converge on the best available scientific evidence 
on issues like climate change. Dynamics of motivated 
cognition would in turn help to explain the ideological 
character of the resulting public controversy over such 
evidence. Many of the emotional associations that drive 
System | risk perceptions, it is posited, originate in (or 
are reinforced by) the sorts of affinity groups that share 
cultural or ideological commitments (Leiserowitz, 2005; 
Sunstein, 2007). Where the group-based associations 
that attach to putative risk sources (guns, say, or nuclear 
power plants) vary, then, we can expect to see systematic 
differences in risk perceptions across members of ideo- 
logically or culturally uniform groups (Weber & Stern, 
2011; Lilienfeld, Ammirati, & Landfield, 2009; Sunstein, 
2006b; Marx, Weber, & Orlove, Leiserowitz, Krantz, 
Roncoli, & Phillips, 2007; Westen, Blagov, Harenski, 
Kilts, & Hamann, 2006). For expositional convenience, 
this account of how heuristic information processing, ide- 
ological predispositions, and motivated reasoning interact 
will be referred to as the “Bounded Rationality Position” 
(BRP). 

An alternative position might see the reasoning-style 
correlates of right-wing ideology as the most consequen- 
tial for disputes over decision-relevant science. Like 
BRP, this position would regard motivated cognition as 
a heuristic-driven form of reasoning. The mental dispo- 
sitions that researchers have identified with conservative 
ideology—dogmatism, need for closure, aversion to com- 
plexity, and the like—are understood to indicate a dis- 
position to rely predominantly on System | rather than 
more effortful System 2 forms of information process- 
ing (Stanovich, 2011, pp. 34-35). Accordingly, the im- 
pact of ideologically motivated cognition, even if not con- 
fined to conservatives, would be disproportionately asso- 
ciated with that ideology by virtue of the negative cor- 
relation between conservativism and the traits of open- 
mindedness and critical reflection that would otherwise 
check and counteract it (Jost, Hennes, Lavine, in press; 
Nam, Jost & van Bavel, 2013). 


Because it predicts that motivated reasoning will be un- 
even across the ideological spectrum, this account can, 
for expositional convenience, be referred to as the “Ideo- 
logical Asymmetry Position” (IAP). Relatively little ex- 
perimental work specifically testing IAP exists at this 
point. It is mixed, with some scholars reporting results 
that suggest conservative subjects display greater resis- 
tance to engaging counter-attitudinal evidence than lib- 
erals do (Nam et al., 2013), but others finding that mo- 
tivated reasoning can skew perceptions—including ones 
formed on the basis of brute sense impressions (Ka- 
han, Hoffman, Evans, Rachlinski, & Braman, 2012)— 
symmetrically across groups of diverse cultural and polit- 
ical orientations (Crawford & Xhambazi in press; Craw- 
ford, 2012). 
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Finally, another position might treat identity-protective 
motivated reasoning as primary in significance. When in- 
dividuals display identity-protective cognition, their pro- 
cessing of information will more reliably guide them to 
perceptions of fact that are congruent with their member- 
ship in ideologically or culturally defined affinity groups 
than to ones that reflect the best available scientific ev- 
idence (Kahan et al., 2011). Nevertheless, this form of 
information processing, when applied to the sorts of facts 
at issue in polarized policy disputes, will often make or- 
dinary individuals better off, in terms of self-interest (Ka- 
han, Peters, Wittlin, Slovic, Ouellette, Braman, & Man- 
del, 2012). Any mistake an individual makes about the 
science on, say, the reality or causes of climate change, 
the disposal of nuclear wastes, or the effectiveness of an 
assault-rifle ban in deterring mass shootings will not af- 
fect the level of risk for her or for any other person or 
thing she cares about: Whatever she, as a single individ- 
ual, does and can do—as consumer, as voter, as partici- 
pant in public discourse—will be too inconsequential to 
have an impact (Downs, 1957). But, insofar as competing 
positions on these issues have come to express member- 
ship in and loyalty to opposing social groups, a person’s 
formation of a belief out of keeping with the one that 
predominates in hers could mark her as untrustworthy or 
stupid, and thus compromise her relationships with others 
(Cohen, 2003). These consequences could substantially 
diminish her welfare—materially and psychically (Sher- 
man & Cohen, 2002; Akerlof & Kranton, 2000; Hillman, 
2010). 


If we imagine that socially adaptive pressures will fa- 
vor reasoning styles that maximize this form of “expres- 
sive utility” (Gigerenzer, 2000), we might, on this ac- 
count, expect the use of more effortful, System 2 forms 
of information processing to magnify, not mitigate, ide- 
ological differences. Individuals disposed to resort to 
heuristic-driven, System 1 cognitive processing should 
not have too much difficulty fitting in: Conformity to 
peer influences, receptivity to elite cues, and sensitiv- 
ity to intuitions calibrated by the same will ordinarily 
guide them reliably to stances that cohere with and ex- 
press their group commitments (Zaller, 1992; Gastil, Bra- 
man, Kahan & Slovic, 2011). But if individuals are adept 
at using more effortful, System 2 modes of information 
processing, then they ought to be even better at fitting 
their beliefs to their group identities. Their capacity to 
make sense of more complex forms of evidence (includ- 
ing quantitative data) will supply them with a special re- 
source that they can use to fight off counterarguments or 
to identify what stance to take on technical issues more 
remote from ones that that figure in the most familiar 
and accessible public discussions (Chen, Duckworth & 
Chaiken, 1999). More importantly still, it will make them 
more likely to understand the significance of competing 
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claims, and related forms of evidence, for the status of 
their group, and thus be more likely to experience un- 
conscious motivations to form identity-congruent assess- 
ments of them. 

This account—which for convenience can be labeled 
the “Expressive Utility Position” (EUP)— inverts the re- 
lationship that BRP posits between motivated cognition 
and dual-process reasoning. Whereas BRP views ideo- 
logical polarization as evidence of a deficit in System 2 
reasoning capacities, EUP predicts that the reliable em- 
ployment of more effortful, conscious information pro- 
cessing will magnify the polarizing effects of identity- 
protective cognition (Kahan, Peters et al. 2012). 

EUP also is inconsistent with IAP. Both BRP and IAP 
assume motivated reasoning reflects a deficit in the ca- 
pacity or disposition to engage in effortful information 
processing of the sort characteristic of System 2. EUP 
(like BRP) is agonistic on whether reliance on heuristic, 
System | reasoning dispositions will be correlated with 
ideological or other values. But because EUP asserts that 
the more effortful, conscious forms of reasoning charac- 
teristic of System 2 magnify identity-protective cognition, 
EUP does not see the correlations featured in IAP as im- 
plying that motivated reasoning should be disproportion- 
ately concentrated in conservatives. 


3 Study: sample, design, and hy- 
potheses 


A study was designed to investigate how heuristic-driven 
information processing, the reasoning-style correlates of 
conservativism, and motivated cognition interact. The 
study included both an observational component, which 
measured the cognitive reasoning dispositions of subjects 
of diverse ideologies; and an experimental one, which as- 
sessed the interaction between cognitive-reasoning dis- 
positions, subjects’ ideologies, and their display of ideo- 
logically motivated reasoning. The design of both com- 
ponents was crafted to pit hypotheses distinctive of BRP, 
IAP, and EUP against one another. 


3.1 Sample 


Subjects for the study consisted of a nationally diverse 
sample of 1750 U.S. adults. The subjects were recruited 
by YouGov, a firm that uses on-line sampling methods 
suited for academic, political, and commercial public 
opinion research. The sample was stratified to reflect 
demographic characteristics, including political outlooks 
and affiliations, representative of the United States gen- 
eral population. The sample was 54% female, and the 
average age of the subjects was 52 years. Seventy-six 
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percent of the subjects were white, and 11% African- 
American. Twenty-eight percent of the sample self- 
identified as Republican, 36% as Democrat, and 30% as 
independent. Twenty-five percent identified themselves 
as either “Liberal” or “very Liberal”: 37% as “Conser- 
vative” or “very Conservative”; and 29% as “Moderate.” 
The mean education level was “some college’; the mean 
annual income was $40,000 to $49,000. The study was 
administered in July 2012. 


3.2 Observational component 


a. Measures. Subjects furnished standard demo- 
graphic data. Party self-identification (““dem_repub’’) was 
measured with a seven-point item (“Strong Democrat, 
Democrat, Independent Lean Democrat, Independent, In- 
dependent Lean Republican, Republican, Strong Repub- 
lican’’). Political ideology (“libcon’’) was measured with 
a five-point item (“Very liberal’; “Liberal”; “Moderate”: 
“Conservative”; “Very Conservative’). 

Responses to these two items (standardized and 
summed (Smith, 2000)) formed a reliable aggregate scale 
(a = 0.82). The scale was constructed so that its cen- 
ter was equivalent to a “neutral” political orientation. 
This was accomplished by centering the standardized 
scale components at their ordinal midpoints (“4” for 
dem_repub, and “3” for libcon) rather than their means 
and centering the composite scale formed by their sum 
at its midpoint as opposed to its mean. Labeled “Con- 
serv_Repub,” the scale was centered at “O” to facilitate 
ease of interpretation, and oriented so that negative val- 
ues denoted a disposition toward Democratic Party affili- 
ation, and liberal ideology and positive values a disposi- 
tion toward Republican Party affiliation and conservative 
ideology. 

Subjects also completed the Cognitive Reflection Test 
(CRT). The CRT is a three-question test that is de- 
signed to measure the disposition to engage in the con- 
scious and effortful form of information processing asso- 
ciated with System 2 as opposed to the heuristic-driven 
form associated with System 1 (Frederick, 2005). A 
performance-based measure, the CRT has been shown to 
be a strong predictor of cognitive biases associated with 
over-reliance on heuristics (Hoppe & Kusterer, 2011)— 
indeed, a stronger one than numerous self-report mea- 
sures of critical reasoning (Toplak, West & Stanovich, 
2011; Liberali, Reyna, Furlan & Pardo, 2011). 

The CRT is a demanding test, and it is not unusual for 
a high proportion of a general population sample to an- 
swer none of the questions correctly (Weller, Dieckmann, 
Tusler, Mertz, Burns & Peters, 2012; Campitelli & Labol- 
lita, 2010). The mean score for subjects in this study was 
0.65 (SD = 0.95). 
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Table 1: Cognitive Reflection Test. “% correct” indicates 
the percentage of the sample answering the indicated item 
correctly. 


Item % correct 





WIDGETS If it takes 5 machines 5 minutes to 
make 5 widgets, how long would 
it take 100 machines to make 100 
widgets? 

A bat and a ball cost $1.10 in 
total. The bat costs $1.00 more 
than the ball. How much does the 
ball cost? 


26% 


BATBALL 13% 


LILYPAD Ina lake, there is a patch of 25% 
lilypads. Every day, the patch 

doubles in size. If it takes 48 days 

for the patch to cover the entire 

lake, how long would it take for 


the patch to cover half of the lake? 


b. Hypotheses. As will be explained presently, re- 
sults from the observational component of the study fur- 
nish maximum insight in conjunction with those of the 
experimental component. However, IAP suggests one 
fairly straightforward hypothesis relating to the observa- 
tional study considered alone: that a “right-wing” outlook 
should be associated with a lower CRT score. 


IAP is based on survey correlations between self- 
identifying as “Republican” or “conservative” and traits 
such as dogmatism, fear of complexity, and need for clo- 
sure (Jost et al., 2003). Because these reasoning traits 
are opposed to reflection and related forms of critical 
thinking, one might expect a right-wing disposition to 
be negatively correlated with performance on the CRT. 
Indeed, researchers who study cognition routinely treat 
CRT scores as a performance-based alternative to or “be- 
havioral” corroborator of the same self-report measures 
featured in research that identifies conservative ideol- 
ogy as distinctively hostile to reflective engagement with 
counter-attitudinal evidence (Stanovich, West & Toplak, 
2011, p. 348 n.5; Pennycook Cheyne, Barr, & Koehler, 
2013; Boschetti, Richer, Walker, Price, & Dutra, 2012; 
Iyer, Koleva, Graham, Ditto, & Haidt, 2012; Pennycook, 
Seli, Koehler, & Fugelsang, 2012). 


Researchers investigating parallel claims about 
religiosity—that it is either a consequence or a cause of 
aversion to complexity, fear of uncertainty, and resis- 
tance to reflectively engaging with counter-attitudinal 
evidence—have reported finding that religiosity is 
negatively correlated with CRT (Pennycook et al., 2013; 
Shenhav, Rand & Greene, 2011; Pennycook Cheyne, 
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Seli, Koehler & Gugelsang, 2012; Gervais & Noren- 
zayan, 2012). A finding that conservative ideology is 
negatively associated with CRT as well would thus help 
corroborate the literature on which IAP is based. 

Only a modest amount of work exists on the relation- 
ship between CRT and political ideology, but it is consis- 
tent with the IAP hypothesis. Two studies report finding 
CRT scores to be slightly lower in self-identified conser- 
vatives than in self-identified liberals (Iyer et al., 2012; 
Pennycook et al., 2012). 

Both of these studies, however, were based on non- 
representative samples. The subjects in the study reported 
in Iyer et al. (2012) were recruited from visitors to a web 
site that features research on the relationship between po- 
litical outlooks and cognitive styles. As an incentive to 
participate, subjects were told they would be supplied 
with a permanently maintained moral personality “pro- 
file” that would enable them to compare their scores on 
various measures to those of other participants, past and 
future. One might expect subjects visiting such a site and 
responding to such an offer to overrepresent highly reflec- 
tive individuals. Indeed, study subjects of all ideologies 
achieved remarkably high mean CRT scores (Iyer et al., 
2012, Table 3)—ones that not only exceeded those ob- 
served in general population samples but that matched or 
exceeded those recorded among students at elite univer- 
sities such as Carnegie Mellon, Princeton, and Harvard 
(Frederick, 2005, Table 1). Thus, one could reasonably 
question whether data from a sample this distinctive sup- 
ports inferences about the relationship between cognitive 
style and political outlooks in the public at large. 

The sample in Pennycook et al. (2012) consisted of 
members of Amzaon.com’s “Mechanical Turk’ work- 
force. Such samples, which are assembled without any 
recruitment or stratification procedures to assure repre- 
sentativeness, have been reported to be highly skewed to- 
ward liberal respondents (Richey & Taylor, 2012). The 
sample in Pennycook et al. (2012, p. 5) reflected such a 
skew.! It is at least possible that a sample drawn from 
this source, members of whom engage in myriad highly 
routinized tasks for an average wage of $1.40 per hour 
(Jaquet, 2011), might underrepresent reflective conser- 
vatives. Researchers have also reported that many Me- 
chanical Turk study subjects have been exposed to ob- 
jective performance measures like those featured in CRT 
on multiple occasions (Chandler, Mueller, & Paolacci, 
2013), an experience that might render their scores un- 
reliable. Finally, there is reason to believe that Mechan- 
ical Turk workers routinely misrepresent their national- 





'Pennycook et al. (2012, p. 5) report that 53% of the subjects in 
their sample self-identified as liberal and 25% identified as conserva- 
tive. Stratified national surveys suggest that approximately 20% of the 
general population self-identifies as liberal and 40% as conservative 
(Gallup, 2012). 
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ity: Researchers report that “substantial” numbers of Me- 
chanical Turk workers recruited for studies open only to 
U.S. “workers” nevertheless participate in them through 
foreign internet service providers (Shapiro, Chandler & 
Mueller, 2013). For these reasons, one might also rea- 
sonably question whether studies based on Mechanical 
Turk workers support reliable inferences about cognitive 
style and ideology in the U.S. general population. 

In sum, the negative correlations between CRT and 
conservativism reported in Iyer et al. (2012) and Penny- 
cook et al. (2012) are definitely plausible and very much 
consistent with the reported correlations between conser- 
vativism and cognitive style in the scholarship that sup- 
ports IAP. But it remains useful to explore the relation- 
ship between CRT and ideology in samples recruited and 
stratified to assure national representativeness. The ab- 
sence of such a finding would be surprising and would 
complicate interpretation of the finding that conserva- 
tivism is negatively associated with various subjective 
measures of heuristic information processing. 

A finding that CRT scores correlate with conservative 
ideology in the general population would have inconclu- 
sive significance for BRP and EUP. BRP does not sug- 
gests any basis to expect an ideological asymmetry in 
the dynamics that result in political conflict over policy- 
consequential facts, but does not necessarily rule it out. 
EUP, in contrast, does assert that such dynamics should 
be symmetric. However, it does not identify the source 
of ideological conflict over fact with the predominance of 
heuristic-driven, System | forms of information process- 
ing. 


3.3. Experimental component 


a. Stimuli and measures. In the experimental compo- 
nent of the study, subjects reported their own perceptions 
of the validity of the CRT upon completion of it. They 
did so by indicating (on a six-point item) their level of 
agreement or disagreement with the statement “I think 
the word-problem test I just took [i.e., the CRT test] sup- 
plies good evidence of how reflective and open-minded 
someone is” (CRT_valid).” 

Subjects responded to this item after being assigned 
to one of three experimental conditions. In the “con- 
trol” condition, subjects were advised simply that “psy- 
chologists believe the questions you have just answered 
measure how reflective and open-minded someone is.” 
In the “skeptic-is-biased” condition, subjects were told 
in addition that “in one recent study, a researcher found 
that people who accept evidence of climate change tend 
to get more answers correct than those who reject evi- 
dence of climate change,” a “finding [that] would imply 





?The wording of the instructions supplied in each experimental con- 
dition is reproduced in the Appendix. 
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that those who believe climate change is happening are 
more open-minded than those who are skeptical that cli- 
mate change is happening.” In contrast, in the “believer- 
is-biased” condition, subjects were advised that “in one 
recent study, a researcher found that people who reject 
evidence of climate change tend to get more answers cor- 
rect than those who accept evidence of climate change,” 
a “finding [that] would imply that those who are skeptical 
climate change is happening are more open-minded than 
those who believe that climate change is happening.” 


b. Hypotheses. Because open-mindedness is a positive 
characteristic, individuals presumably have an emotional 
stake in believing that people who subscribe to their own 
ideology are more open-minded and reflective, or at least 
not less, than those who subscribe to an opposing one. 
Here subjects were presented evidence relevant to that is- 
sue: the respective performance on an “open-mindedness 
test” of people who either accepted or rejected a posi- 
tion strongly associated with membership in the subjects’ 
own ideological groups. The subjects were in fact sup- 
plied relatively spare information about the validity of 
CRT: only the representation that psychologists view it 
as valid, plus the subjects’ own experience in having just 
taken it. The prospect that they would engage in moti- 
vated reasoning, though, supplied a basis for believing 
they would treat that evidence as establishing the test’s 
validity conditional on whether doing so would gratify or 
disappoint their stake in believing that members of their 
ideological group were more open-minded than members 
of an opposing one (Cohen, 2003). 

Ata very concrete level, this design can be viewed as a 
model of how ordinary people process information about 
studies of the ideological correlates of cognitive reason- 
ing styles (e.g., Mooney 2012). Such readers will have 
little to go on besides scholars’ or commentators’ rep- 
resentations that the tests of open-mindedness featured 
in such studies are valid. If such readers are inclined to 
credit such representations only when the studies’ results 
gratify their interest in forming and maintaining the be- 
lief that people who share their own ideology are more 
open-minded, then their assessments of that research will 
itself be biased by ideologically motivated reasoning. 

But, abstracting from the particulars, the study design 
can be also thought of as modeling how ideologically mo- 
tivated reasoning might bias considerations of empirical 
evidence generally. On policy debates over matters as 
diverse as climate change, gun control, the death penalty, 
and fiscal policy, ordinary citizens are presented with evi- 
dence, often in the form of second-hand characterizations 
of the findings of “scientific studies”’. If their assessments 
of the validity of such evidence is conditional on its fit 
with their ideological predispositions, then citizens will 
not converge on the best available evidence but rather will 
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polarize on policy-relevant facts (Lord, Ross & Lepper 
1979; Kahan, Jenkins-Smith & Braman, 2011). 


BRP, IAP, and EUP all predict motivated reasoning in 
this study. They generate different hypotheses, however, 
about the form that such reasoning will take. 


IAP predicts that the observed motivated reasoning 
should be stronger in conservative subjects. Reflect- 
ing their disposition toward dogmatism and closed- 
mindedness, relatively right-wing subjects should be 
more inclined to see CRT as a valid test in the believer-is- 
biased condition than in the skeptic-is-biased condition. 
Scholars who have presented evidence of the cognitive- 
style correlates of ideology depict more liberal or left- 
wing individuals as more open-minded and reflective and 
thus less subject to motivated cognition. Accordingly, 
in the study, this work would predict that relatively left- 
wing subjects’ assessments of the validity of CRT should 
be comparable in both the skeptics-biased and believer- 
is-biased condition. This finding would be a particu- 
larly compelling affirmation of IAP, moreover, in con- 
junction with a finding in the observational component 
of the study that right-wing dispositions correlate with a 
lower CRT score. 


BRP understands motivated reasoning to be an arti- 
fact of the disposition to use low-level, System 1 infor- 
mation processing. Unlike IAP, BRP does not (or does 
not necessarily) predict that motivated reasoning will be 
ideologically asymmetric. But BRP does predict that it 
will be higher among subjects who score relatively low 
in CRT than it will be in those who score relatively high. 
On this account, any ideological polarization observed 
in the skeptic-is-biased and believer-is-biased conditions 
should dissipate as CRT increases. 

EUP supports predictions strikingly different from 
those of either IAP or BRP. Not only does EUP fail 
to predict ideological asymmetry in motivated reason- 
ing. It predicts that the tendency of both right-wing and 
left-wing subjects to form ideologically congruent as- 
sessments of the “validity” of CRT will increase with 
their CRT scores. A// subjects, EUP posits, will experi- 
ence psychological pressure to fit their perceptions of the 
CRT’s validity to their interest in believing that members 
of their group are more open-minded about evidence on 
climate change. Nevertheless, their success in achieving 
this end will depend on their comprehension of the ques- 
tions being posed and their appreciation of what differing 
answers signify about the open-mindedness of individ- 
uals who share their ideologies. Because subjects who 
are high in CRT assess information more methodically 
and reflectively, they are less likely to misunderstand the 
question, and thus less likely to avoid the unconscious 
pressure to fit their assessments of the evidence at hand 
to the conclusion that fits their expressive interests. 
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Table 2: Ordered logistic regression analysis of CRT 
scores. N = 1750. Outcome variable is CRT score. Pre- 
dictor estimates are ordered-logit coefficients with z-test 
statistic indicated parenthetically. Bolded typeface indi- 
cates predictor coefficient, model LR”, or incremental 
change in model LR? is significant at p < 0.05. 








Model 1 Model 2 
Conserv_Repub 0.07 (1.45) 0.05 (0.97) 
male 0.68 (6.60) 
white 0.72 (5.41) 
education 0.31 (8.39) 
income 0.06 (3.32) 
religiosity —0.15 (—3.63) 
LRx? 2.1 (1 df) 261.9 (6 df) 
ALRx? 259.8 (5 df) 


3.4 Statistical power and missing data 


Certain of the competing hypotheses associated with 
BRP, IAP, and EUP turned on whether or not an obser- 
vational correlation or experimental effect would be ob- 
served. The strength of inferences drawn from “null” 
findings depends heavily on statistical power. The large 
size of the sample furnished adequate power to detect 
even small effect sizes (e.g., r = 0.10) with a probabil- 
ity well over 0.80 at p < .05 (Cohen, 1988). As a result, 
findings of nonsignificance could be equated with lack of 
effect with low risk of Type II error (Streiner, 2003). To 
assure full exploitation of the power associated with the 
large sample size, missing data were replaced by multi- 
ple imputation (King, Honaker, Joseph & Scheve, 2001; 
Rubin, 2004). 


4 Results 


4.1 Observational component 


Subjects’ CRT scores were regressed (Table 2) against 
political orientations and other individual characteristics 
known to be associated with CRT performance, including 
gender, race, education, income, and religiosity (which 
was measured with a composite scale that aggregated 
self-reported church attendance, frequency of prayer, and 
perceived importance of God, a = 0.82). The coefficient 
for Conserv_Repub was positive—indicating that CRT 
scores increase as subjects become progressively more 





3The experiment was first analyzed without imputed data, using list- 
wise deletion for observations containing missing data (Kahan 2012a). 
The results did not differ materially from those reported. 
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right-wing in their orientation—but nonsignificant both 
when treated as a zero-order predictor (b = 0.07, p = 0.15) 
and when included in a model that controlled for other 
characteristics (b = 0.05, p = 0.33). The other charac- 
teristics, in contrast, all showed significant, independent 
effects on CRT scores (Table 2). 

The Conserv_Repub scale is a more reliable measure 
of the subjects’ ideological dispositions than either of its 
components alone, and thus supports more reliable infer- 
ences about the relationship between political outlooks 
and CRT scores (Gliem & Gliem, 2003). Nevertheless, 
to promote comparability of the results reported in this 
study and ones that use only a single political orientation 
measure, the relationships between subjects’ scores and 
aspects of each component of Conserv_Repub were also 
examined. 

The correlation with self-reported liberal-conservative 
ideology (libcon) was negative—indicating a decline in 
score aS conservativism increases. The size of the ef- 
fect, however, was only trivially different from zero and 
statistically nonsignificant (r, = -0.02, p = 0.45). When 
the subjects who self-identified as “Moderates” were ex- 
cluded, and the remainder split into groups who identified 
as either “Very liberal” or “Liberal,” on the one hand, or 
“Very conservative” or “Conservative,” on the other, there 
was a slightly larger but still statistically nonsignificant 
difference (AM = 0.09, t = 1.41, p = 0.16) in the mean 
scores of “liberals” (M = 0.75, SEM = 0.05) and “conser- 
vatives” (M = 0.67, SEM = 0.04). 

The correlation with self-reported party affiliation 
(rs = 0.08, p < 0.01) was positive and significant, in- 
dicating that CRT scores increased with the strength 
of subjects’ identification with the Republican party on 
dem_repub, the 7-point measure of partisan identifica- 
tion. When subjects who self-identified as either “Inde- 
pendents” or “Independents” who “lean” either Demo- 
crat or Republican were removed, and the remainder split 
into ones who identified as either “Democrat” or “Re- 
publican,” the difference in the mean score of “Repub- 
licans” (M = 0.66, SEM = 0.04) and “Democrats” (M = 
0.52, SEM = 0.03) was also statistically significant (AM 
= 0.14, t = 2.48, p < 0.05). The gap between “Re- 
publicans” (M = 0.71, SEM = 0.04) and “Democrats” 
(M = 0.59, SEM = 0.03) remained statistically signifi- 
cant (AM = 0.13, t = 2.54, p < 0.05) when the scores 
of subjects who identify as “Independents” but “lean” to- 
ward one or the other of the major parties were treated as 
partisans (Petrocik, 2009). 

Based on work relating personality-trait and self- 
reported measures of cognitive style to ideology, IAP pre- 
dicted that CRT scores would be negatively correlated 
with right-wing ideology. This hypothesis was not con- 
firmed. 


414 


Ideology, motivated reasoning, and cognitive reflection 


4.2 Experimental component 


a. Ideologically motivated reasoning generally. Con- 
sistent with BRP, IAP, and EUP, subjects displayed ideo- 
logically motivated reasoning in their assessments of the 
validity of the CRT (Figure 1, Panel (A)). In the control 
condition, right-wing and left-wing subjects (identified 
by their scores relative to the mean on Conserv_Repub), 
formed comparable judgments of the validity of the CRT 
as a measure of how “reflective and open-minded” peo- 
ple are (M = 3.0, SEM = 0.2, for both; AM = 0.08, 
t = 0.59, p = 0.56). Those assigned to the “skeptic- 
is-biased” condition, in contrast, divided on ideological 
lines: right-wing subjects were Jess inclined to treat the 
CRT as valid (M = 2.4, SEM = 0.2), and left-wing ones 
more inclined to do so (M = 3.5, SEM = 0.2), when 
told that climate-change skeptics generally score lower 
on the test. This effect was reversed in the “believer- 
is-biased” condition: told that climate skeptics generally 
score higher on the test, relatively right-wing subjects 
were now more inclined (M = 3.1, SEM = 0.2), and rel- 
atively left-wing ones less, to judge the CRT to be valid 
(M = 2.6, SEM = 0.2). 


b. Symmetry. Figure 1 plots subject responses using 
a locally weighted regression smoother. This technique, 
which traces the profile of the data in detail without mak- 
ing any particular assumptions about their distribution, is 
an appropriate device both for graphical summary assess- 
ments of the raw data and for determining the appropriate 
form (linear or curvilinear) of the statistical model to ap- 
ply in analyzing them (Jacoby, 2000). 

Visual inspection of Figure | does not suggest that 
the observed motivated-reasoning effect increased as sub- 
jects became more right-wing. Indeed, liberal Demo- 
cratic subjects were inclined to discount the validity of 
the CRT in the “believer-is-biased” condition relative to 
the “‘skeptic-is-biased” condition (AM = 0.86, t = 6.37, 
p < 0.01) by an amount slightly but nonsignificantly 
larger (AM = 0.11, t = 0.57, p = 0.57) than the amount 
by which conservative Republican ones were inclined to 
discount it in “skeptic-is-biased” condition relative to the 
“believer-is-biased” one (AM = 0.75, t = 5.39, p <0.01).4 





4To perform a more exacting test of whether the motivated-reasoning 
effect observed in the experiment was symmetric or asymmetric with re- 
spect to subjects’ ideological dispositions, a quadratic regression model 
was fit to the data. Formed by adding to Model 1 of Table 3 terms 
that squared the value of the subjects’ scores on Conserv_Repub, the 
resulting model reflected the assumption that the motivated-reasoning 
elicited by the experimental treatment would be stronger as subjects’ 
outlooks approached one extreme of the disposition measured by Con- 
serv_Repub (either the “right-” “or the “left-wing” end) than it would be 
as that disposition approached the opposite extreme. The added terms 
had coefficients that were trivially different from zero and nonsignifi- 
cant, and the addition of them did not result in a significant improve- 
ment in fit (Kahan 2012a). 
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Figure 1: Graphical summary of experimental results. Locally weighted regression, applied separately for each exper- 
imental condition, plots the relationship between the political outlooks and responses to CRT_valid. Panel (A) plots 
responses for all study subjects. Panels (B), (C), and (D) plot results only for study subjects with CRT scores of 0, 1, 
and 2 or 3, respectively. Conserv_Repub, the scale used to measure the subjects’ political outlooks, is centered at the 
point corresponding to a subject who self-identified as a “moderate” on the 5-point liberal-conservative ideology scale 
and as an “Independent” (who declined when “pushed” to “lean” toward either party) on the 7-point partisan-self- 
identification scale. Individuals who identified themselves as either “liberal” and “Democrat” or as “conservative” and 
“Republican” would have scored -0.95 and 0.95 on Conserv_Repub, respectively. The extreme values on the scale— 
-1.65 and 1.65, respectively—correspond to the scores of individuals who identified themselves as “Very liberal” and 


“Strong Democrat” and “Very conservative” and “Strong Republican,” respectively. 
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c. Interaction with CRT. The final experimental hy- 
potheses were BRP’s prediction that ideological polariza- 
tion should abate as individuals become more disposed to 
use System 2 processing and EUP’s prediction that mo- 
tivated reasoning would instead increase. As is evident 
in Panels (B)—(D) of Figure 1, the tendency of subjects 
(varying in political dispositions) to form ideologically 
congenial assessments of the validity of the Cognitive Re- 
flection Test became progressively more pronounced as 
subjects’ CRT scores increased. The effect appears most 
dramatic in the “believer-is-biased” condition. 


To confirm and test the statistical significance of this 
effect, a regression model reflecting the interaction of 
CRT and subjects’ ideological dispositions was fit to 
the data. The results are reported in Table 3, Model 
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(D) CRT >1 


2. This model fit the data significantly better than 
one lacking terms reflecting the hypothesized interac- 
tion (ALR y2 = 12.64(6), p = 0.01). The coefficient 
for the interaction term added to assess the relationship 
between CRT scores and assignment to the “believer-is- 
biased” condition (Con_Rep_x_zCRT_x_believer), was 
positive and significant (b= 0.30, p < 0.01). Thus, 
in the “believer-is-biased” condition, subjects of op- 
posing ideologies polarized to a greater extent as their 
CRT scores increased. The coefficient for the interac- 
tion term added to assess the relationship between CRT 
scores and assignment to the “skeptic-is-biased” con- 
dition (Con_Rep_x_zCRT_x_skeptic) was only trivially 
different from zero and nonsignificant (b = —0.03, p = 
0.76). Thus, in the “skeptic-is-biased” condition, the de- 
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Table 3: Ordered logistic regression analysis of exper- 
imental component. N = 1750. Outcome variable is 
CRT_valid. Predictor estimates are ordered-logit co- 
efficients with z-test statistic indicated parenthetically. 
Bolded typeface indicates predictor coefficient, model 
LRy?, or incremental change in model LRx? is signif- 
icant at p < 0.05. “Skeptic-is-biased” and “Believer-is- 
biased” are dummy variables that reflect the experimen- 
tal assignment (unassigned = 0; assigned = 1). CRT score 
(“zCRT’”) and Conserv_Repub are both centered at 0 to 
promote ease of interpretation. 


























Model 1 Model 2 

Conserv_Repub 0.12 (—1.60) —0.12 (—1.58) 
Skeptic-is-biased 0.02 (—0.29) —0.03 (—0.27) 
Believer-is-biased 0.18 (—1.74) —0.17 (—1.63) 
Con_Rep_x_skeptic —0.50 (—4.63) —0.50 (—4.62) 
Con_Rep_x_believer 0.38 (3.59) 0.37 (3.47) 
zCRT 0.04 (0.53) 
C_R_x_zcrt —0.05 (—0.76) 
zCRT_x_skeptic —0.01 (—0.12) 
zCRT_x_believer —0.07 (—0.65) 
Con_Rep_x_zCRT_x_skeptic —0.03 (—0.31) 
Con_Rep_x_zCRT_x_believer 0.30 (2.95) 
LRy? 89.96 (5 df) 102.60 (11 df) 
ALRy? 12.64 (6 df) 


gree of ideological polarization did not meaningfully vary 
in relation to subjects’ CRT scores. 


Graphic illustration enables practical assessment of 
this interaction effect. Figure 2 plots the predicted prob- 
ability that subjects of opposing political outlooks with 
either “low” or “high” in CRT scores would agree (ei- 
ther slightly, moderately, or strongly) that the CRT is 
a valid measure of open-mindedness in each of the ex- 
perimental conditions. The estimates were formed us- 
ing regression Model 2 in Table 3. For “Liberal Demo- 
crat” and “Conservative Republican,” the value assigned 
to Conserv_Repub was set at the level a subject would 
have scored had she selected either “liberal” and “Demo- 
crat” or “conservative” and “Republican” on the 5-point 
liberal-conservative ideology and 7-point partisan self- 
identification items, respectively. For “low CRT,” the 
value assigned to the CRT predictor was 0 correct, a score 
that would place those individuals in the bottom 60% 
within the sample. For “high CRT,” the predictor was 
set at 2 correct, a score matched or exceeded by 20% of 
the sample. 
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As can be seen, the impact of CRT performance on 
motivated reasoning is concentrated in the “believer-is- 
biased” condition. The difference in predicted probability 
of judging the CRT to be “valid” in that condition is 28 
percentage points (+ 19%, CL = 0.95) greater for high- 
CRT partisans than it is for low-CRT partisans, whose 
likelihoods of judging the CRT to be “valid” do not mean- 
ingfully differ in that condition. Both high-scoring and 
low-scoring partisans are highly likely to disagree about 
the validity of the CRT in the “skeptic-is-biased” condi- 
tion. The likelihood that high-scoring ones will disagree 
is higher than is the likelihood that low-scoring ones in 
that condition will, but the difference is modest and not 
significant at 0.95 level of confidence (8%, + 11%). 

These results are more consistent with EUP than BRP. 
Whereas the former predicted that motivated reasoning 
would abate as CRT increases, the latter predicted that 
it would not but would instead intensify. Such intensi- 
fication did occur, but only in one of the experimental 
conditions. 


5 Discussion 


5.1 Summary of results 


The motivation (consciously, at least) for this study was 
to assess three dynamics understood to be relevant to po- 
litical contestation over facts that turn on scientific or re- 
lated forms of empirical evidence: heuristic-driven rea- 
soning, the cognitive-style correlates of political conser- 
vativism, and motivated cognition. For that purpose, the 
study measured the cognitive reflection of individuals of 
diverse ideologies and also tested how willing they were 
to entertain evidence that those who disagreed with their 
ideological group’s position on a charged issue—climate 
change—were themselves open-minded and reflective. 

The study considered three conjectures about how 
heuristic-driven reasoning, ideological predispositions, 
and motivated reasoning relate to one another. The first 
the “Bounded Rationality Position” or BRP—asserts that 
public conflict over risk and other policy relevant facts is 
a consequence of the predominance of heuristic-driven, 
System | information processing, which interferes with 
the public’s understanding of complicated evidence and 
motivates it to assess evidence consistently with cultural 
or ideological predispositions (e.g., Marx et al. 2007; We- 
ber & Stern, 2011). 

The study did not support BRP. On the contrary, the ex- 
perimental component of the study demonstrated that the 
disposition to engage in conscious and effortful System 
2 information processing—as measured by the Cognitive 
Reflection Test (CRT)—actually magnifies the impact of 
motivated reasoning. 
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Figure 2: Interaction between CRT and experimental treatment. Derived via Monte Carlo simulation (King, Tomz & 
Wittenberg, 2000) from regression model reported in Table 3, Model 2. Point estimates indicate predicted probability 
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and “high CRT” are 0 and “2,” respectively. Cls reflect 0.95 level of confidence. 
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The study results were also inconsistent with the “Ide- 
ological Asymmetry Position” or IAP. IAP attributes pub- 
lic conflict over policy-relevant facts to an affinity be- 
tween conservative ideology and traits such as dogmatism 
and aversion to complexity, which fuel the motivated re- 
jection of evidence incompatible with conservative policy 
aims (Jost et al., 2003). 

The study findings pose two distinct challenges to IAP. 
First, the study found no meaningful correlation between 
right-wing ideology and CRT performance, an objective 
measure of reflection that has been shown to be one of the 
strongest predictors of conscious, effortful information 
processing. The CRT is routinely treated as a “stronger” 
performance-based alternative to (Pennycook et al. 2013) 
or “behavioral” corroborator of (Iyer et al., 2012) Need 
for Cognition, Need for Closure, dogmatism, fear of com- 
plexity and like measures (Stanovich, West & Toplak, 
2011; Boschetti et. al., 2012), which are the basis of the 
research that finds conservatives to be distinctively resis- 
tant to counter-attitudinal evidence. The CRT has been 
used to test and generate evidence supportive of the par- 
allel claim that religiosity is associated with resistance to 
scientific and other forms of empirical evidence. It is thus 
puzzling that CRT did not correlate negatively with con- 
servativism (and indeed correlated positively with self- 
identification with the Republican Party) in the large na- 
tionally representative sample used in this study. This 
result is at odds with those based on nonrepresentative 
samples. 

Second, this study used an experimental design and a 
Statistical-testing strategy specifically fitted to evaluating 
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the assumption that the force of ideological motivated 
reasoning varies in intensity across the right-left polit- 
ical spectrum. It found that, when assessing evidence 
of the other group’s propensity to consider evidence in 
an open-minded and reflective way, liberals and conser- 
vatives were uniformly prone to ideologically motivated 
reasoning. This result reinforces those observed in other 
studies that have reported finding that motivated reason- 
ing is Symmetric across groups of opposing political and 
cultural outlooks.° 

The study also examined one other account of political 
polarization over risk and related policy-relevant facts: 
the “Expressive Utility Position” or EUP. The study re- 
sults were largely in accord with this view. 

EUP alone predicted both that ideologically motivated 
reasoning would be symmetric and that it would be am- 
plified by higher CRT scores. Those hypotheses reflect a 
theory that sees ideologically motivated cognition not as a 





5In order for this result to support the inference that motivated rea- 
soning is symmetric with respect to ideology in general, it must be 
assumed that that “liberal Democrats” and “conservative Republicans” 
have a comparable identity-protective stake in their positions on the par- 
ticular issue on which subjects were exposed to counter-attitudinal evi- 
dence in this study. If, say, liberal Democrats are more intensely com- 
mitted to being perceived as “open-minded” and “reflective” on climate 
change than conservative Republicans, then one might expect the for- 
mer to feel more threatened by counter-attitudinal evidence and hence 
more pressured to dismiss such evidence in a reflexive or dogmatic fash- 
ion. The design of the study cannot rule this possibility out. But while 
such an explanation would “fit” the results of the study, it would not be 
particularly supportive of the “asymmetry thesis” premise (Jost et al., 
2003; Nam et al., 2013) that conservatives are peculiarly vulnerable to 
ideologically motivated reasoning. 
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reasoning deficiency but as a reasoning adaptation suited 
to promoting the interest that individuals have in con- 
veying their membership in and loyalty to affinity groups 
central to their personal wellbeing. Because individuals 
make use of this form of information-processing to pro- 
tect their stake in all manner of groups—including ones 
not connected to politics (Sherman & Cohen, 2006; Has- 
torf & Cantril, 1954)—there is no reason to expect it to 
be more pronounced among people who subscribe to any 
particular ideology. In addition, because this dynamic re- 
liably guides individuals to assign information the signif- 
icance that advances their personal ends (albeit not their 
collective ones (Kahan, 2012b)), there is good reason to 
expect it to be used with even greater precision and ac- 
curacy by individuals disposed to engage in conscious, 
effortful forms of information processing. 

The only discrepancy between the study results and 
EUP was the limited scope of the interaction between 
cognitive reflection and motivated reasoning. Higher 
CRT scores meaningfully amplified motivated reasoning 
in only one of the two experimental conditions that pre- 
sented evidence that was expected to polarize (and did 
polarize) subjects of opposing political outlooks. 

While not hypothesized, this result is not incompati- 
ble with EUP. The charge that “climate-change skeptics” 
are “closed minded” and “biased” is likely more famil- 
iar to people than the charge that “climate-change be- 
lievers” are. The wording of the information furnished 
to subjects was in fact fairly dense. If subjects low in 
CRT had difficulty with it or were reluctant to expend 
the mental effort to parse it, they would have been more 
likely to misunderstand it when it conveyed the less fa- 
miliar claim (“believers are closed minded’) than when 
it conveyed the more familiar one (“skeptics are closed 
minded”). Subjects with higher CRT scores were more 
likely to grasp the surprising meaning conveyed by the 
instruction in the “believer-is-biased” condition, making 
them more prone than low-scoring ones in that condition 
to form an appraisal of the test’s validity that conformed 
to their ideological predispositions. 


5.2 Implications for ongoing study of dual- 
process and motivated reasoning 


The status of motivated reasoning within dual process 
reasoning theories has not been studied extensively by so- 
cial psychologists. Many commentators have assumed— 
not implausibly—that ideologically motivated cognition 
is a manifestation of unconscious, heuristic-driven rea- 
soning processes amenable to being overridden by dis- 
positions that promote reflection and critical engagement 
with counter-attitudinal evidence (e.g., Lilienfeld et al. 
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2009; Sunstein, 20006b; Westen et al., 2006). The results 
of this study cast doubt on this claim. 

It should be stressed, however, that this result is not 
in tension with dual-process reasoning theories generally, 
but only with a particular surmise about how a particu- 
lar cognitive dynamic should be assimilated to such the- 
ories. Indeed, in work that actually predates the adap- 
tation of the now-familiar “System 1/System 2” frame- 
work, at least some dual-process theorists took the posi- 
tion that motivated reasoning should be expected to char- 
acterize higher-level, systematic forms of cognition as 
well, particularly when the motivated use of conscious 
and effortful processing could be expected to advance an 
agent’s ends in maintaining his or her connection to an 
identity-defining group (Cohen, 2003; Chen & Chaiken, 
1999; Chaiken & Maheswaran, 1994; Giner-Sorrolla & 
Chaiken, 1997). The current study, by generating evi- 
dence that the disposition to engage in effortful informa- 
tion processing can amplify motivated reasoning, is more 
consistent with the latter position. 

Indeed, these findings informed the “expressive util- 
ity position” tested in the study. That same hypothe- 
sis, moreover, figured in a previous observational study, 
which found that cultural polarization over climate- 
change and nuclear-power risks are greatest among in- 
dividuals who are higher in scientific literacy and in nu- 
meracy, a technical-reasoning measure of which CRT is 
a subcomponent (Kahan, Peters et al. 2012). The ex- 
perimental results of the present study help corroborate 
the inference that the correlations reported in that earlier 
study are property attributed to the power of motivated 
cognition to penetrate the forms of information process- 
ing associated with Kahneman’s “System 2” form of rea- 
soning. 

Of course, the findings of this single study do not 
conclusively demonstrate that the disposition to engage 
in reflective rather than heuristic-information processing 
invariably magnifies ideologically motivated reasoning. 
Empirical studies, when valid, merely supply more evi- 
dence to believe or disbelieve a hypothesis, the truth of 
which must be assessed on the basis of all the valid ev- 
idence at hand and thereafter reassessed on the basis of 
any evidence generated by future valid studies (Popper, 
1959). 

Specific to this particular study, there are many rea- 
sons not to consider the issue of ideologically moti- 
vated reasoning and reflective information-processing to 
be “closed”. The most important of these is continuing 
debate over the adequacy of existing measures of reflec- 
tive information-processing and critical-thinking disposi- 
tions generally. One issue concerns the relative validity 
of self-reporting measures (such as “Need for Cognition” 
and “Need for Closure”), on the one hand, and objective 
measures such as the CRT, on the other. 
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Another concerns the strength of CRT relative to al- 
ternative objective measures. Because CRT is such a de- 
manding test—because it is not uncommon, in a general 
population sample, for over a third of the sample to get 
zero answers correct, and for less than 10% to answer 
all three questions correctly—CRT is of limited value in 
explaining variance among a large portion of the popula- 
tion. Numeracy is superior to CRT in this respect, and is 
a stronger predictor of vulnerability to cognitive biases, 
likely for this reason (Liberali et al., 2011; Weller et al., 
2012). But as indicated, numeracy too has been found 
to predict greater, not diminished, political polarization 
over risks like the ones associated with climate change 
and nuclear power (Kahan, Peters et al., 2012). 


Exactly what both CRT and numeracy measure, how- 
ever, is also contested (Liberali et al., 2011). In partic- 
ular, whether either measures critical reflection directly 
or instead only indirectly—as a correlate of a more gen- 
eral proficiency in mathematical computation or even a 
general form of intellectual capacity—continues to be 
debated. Some researchers have concluded that CRT 
is diagnostic of all manner of critical thinking (Toplak, 
West & Stanovich, 2011), including the motivation to re- 
examine existing beliefs (Campitelli & Labollita, 2010; 
Stanovich, West & Toplak, 2011) associated with the 
Actively Open-minded Thinking construct (AOT; Baron, 
2008) or reflection/impulsivity (Baron, Badgio & Gask- 
ins, 1986). But whether CRT indeed measures willing- 
ness to revise strongly held beliefs, or does so as well as a 
measure specifically geared toward AOT, is not resolved. 
Necessarily unresolved, too, then, is whether a study of 
motivated reasoning that investigated its interaction with 
a suitable AOT scale (e.g., Haran, Ritov & Mellers, 2013) 
would find that the form of critical thinking that scale 
measures likewise magnifies ideologically motivated rea- 
soning in the manner that CRT, numeracy, science liter- 
acy (Kahan, Peters et al., 2012), and political knowledge 
(Arceneaux & Johnson, 2013; Taber & Lodge, 2006) all 
do or instead diminishes it (Baron, 1995). 


It thus makes sense to continue intensive study of the 
relationship between ideological polarization and infor- 
mation processing using a variety of measures, a strategy 
that recognizes the role that convergent validity always 
plays in insuring against the necessary imperfection of 
any single measure or method in the social sciences. In- 
deed, if a variety of measures and methods do converge 
on the conclusion that ideologically motivated reason- 
ing is impervious to the habits of mind most strongly 
associated with valid, evidence-based inference, such a 
program of study is the one most likely to broaden in- 
sight into how the conditions that generate an influence 
this inimical to reason might be counteracted (Fernbach, 
Rogers, Fox & Sloman, 2013; Lavine, Johnston & Steen- 
bergen, 2012). 
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5.3. Implications for counteracting ideolog- 
ically motivated reasoning 


The goal of empirically investigating the sources of polit- 
ical conflict over risk and other policy-consequential facts 
is not merely to explain this phenomenon but also to aid 
in discovery of devices for mitigating it. The study de- 
scribed in this paper makes a contribution to that end as 
well. 

It does this primarily by helping to inform hypothe- 
ses about how such dynamics might be combated. Many 
scholars have suggested “debiasing” strategies aimed at 
correcting the distorting effect of System 1 reasoning on 
public perceptions of risk (Lilienfeld et al., 2009; Jolls 
& Sunstein, 2006). Because such distortions are real— 
and substantially interfere with human wellbeing in myr- 
iad domains—pursuit of System | debiasing techniques is 
unquestionably important. Nevertheless, if, as the present 
study implies, ideologically motivated cognition is not 
a consequence of an over-reliance on heuristic reason- 
ing, then System | debiasing strategies should not be ex- 
pected to abate polarization over climate change, nuclear 
power, gun control, the HPV vaccine or like issues (Ka- 
han, Slovic, Braman & Gastil, 2006). 

What is needed instead are interventions that remove 
the expressive incentives individuals face to form percep- 
tions of risk and related facts on grounds unconnected to 
the truth of such beliefs (Lessig, 1995). Extending the 
analysis of previous papers, this one has suggested that 
ideologically motivated reasoning is in fact expressively 
rational at the individual level, because it conveys indi- 
viduals’ membership in and loyalty to groups on whom 
they depend for various forms of support, emotional, ma- 
terial, and otherwise (Hillman, 2010; Akerlof & Kranton, 
2000). 

This account, however, presupposes that beliefs on 
risks and related facts bear social meanings—that they 
are, in fact, generally understood (tacitly, at least) to con- 
vey that the individuals who espouse them are committed 
to one group rather than another (Cohen, 2003). Not all 
risks and policy-relevant facts have this quality; indeed, 
relatively few do, and on the vast run of ones that do not 
(e.g., that pasteurization removes infectious agents from 
milk; that fluoridation of water fights tooth decay; that 
privatization of the air-traffic control system is inimical 
to air safety), we do not observe significant degrees of 
ideological or cultural polarization. 

There is little reason to believe, moreover, that the 
meanings of highly contested facts are insusceptible of 
revision in a manner that would disconnect particular 
positions on them from membership in identity-defining 
groups. One can understand the historical shift in public 
opinion toward the risks posed by cigarettes (including 
third-party ones from passive smoke exposure or from the 
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societal expenditures necessary to treat individuals with 
lung cancer) as having been mediated by informational 
campaigns aimed at altering the positive meanings that 
dismissing evidence of the health hazards of smoking ex- 
pressed in certain subcommunities (United States Public 
Health Service, 2000; Gusfield 1993). 


The expressive account of ideological polarization, 
then, underscores the value of forming and testing hy- 
potheses about how to regulate the social meaning of 
risks and related policy-relevant facts. Indeed, research 
focusing on forecasting techniques for identifying tech- 
nologies vulnerable to polarizing meanings, on govern- 
mental processes for protecting the “science commu- 
nication environment” from influences that cause such 
meanings to take hold, and on framing and other strate- 
gies for cleaning up that environment once it has been 
contaminated with polarizing meanings (Kahan, 2012b), 
is already well underway (Corner, Whitmarsh & Xe- 
nias, 2012; Druckman & Bolsen, 2012, 2011, Nisbet & 
Scheufele, 2009). 


5.4 Conclusion: The tragedy of the science 
communications commons 


What might be characterized as the “expressive rational- 
ity” of ideologically motivated reasoning (Kahan, Peters 
et al., 2012) is intrinsic to a collective action problem 
(Lessig, 1995). When societal risks become suffused 
with antagonistic social meanings, it is (often if not al- 
ways, and with respect to many if not all issues) individu- 
ally rational for ordinary members of the public to attend 
to information in a manner that reliably connects them to 
the positions that predominate in their identity-defining 
groups. Nevertheless, if ideologically diverse individuals 
all follow this strategy simultaneously, they will be col- 
lectively worse off, since under these conditions, demo- 
cratic institutions are less likely to converge, or to con- 
verge as rapidly as they otherwise would, on policies that 
reflect the best available evidence on how to protect ev- 
eryone from harm. But because what any ordinary in- 
dividual believes about policy will not make a difference, 
the collective irrationality of ideologically motivated rea- 
soning does not by itself create any reliable pressure or 
mechanism to induce individuals to process information 
in a different, and morally and politically superior, way. 


Overcoming ideological polarization over societal 
risks and related facts thus demands collective action 
specifically geared at dissolving this “tragedy of the 
science communications commons” (following Hardin, 
1968). Supplying the knowledge needed to guide such 
action is a contribution that decision science is uniquely 
poised to make. 
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Appendix. Experimental stimuli 


A. RANDOMLY ASSIGN to condition 1 or 2 
or 3. 


1. Control condition 


Psychologists believe the questions you have just an- 
swered measure how reflective and open-minded some- 
one is. 


2. Skeptic-is-biased condition 


Psychologists believe the questions you have just an- 
swered measure how reflective and open-minded some- 
one is. 

In one recent study, a researcher found that people 
who accept evidence of climate change tend to get more 
answers correct than those who reject evidence of cli- 
mate change. If the test is a valid way to measure open- 
mindedness, that finding would imply that those who be- 
lieve climate change is happening are more open-minded 
than those who are skeptical that climate change is hap- 
pening. 


3. Believer-is-biased condition 


Psychologists believe the questions you have just an- 
swered measure how reflective and open-minded some- 
one is. 

In one recent study, a researcher found that people 
who reject evidence of climate change tend to get more 
answers correct than those who accept evidence of cli- 
mate change. If the test is a valid way to measure open- 
mindedness, that finding would imply that those who are 
skeptical that climate change is happening are more open- 
minded than those who believe climate change is happen- 
ing. 


B. ALL CONDITIONS CRT_valid 


How strongly do you agree or disagree with this state- 
ment? [strongly Disagree, moderately disagree, slightly 
disagree, slightly agree, moderately agree, strongly 
agree] 


I think the word-problem test I just took sup- 
plies good evidence of how reflective and open- 
minded someone is. 


Ideology, motivated reasoning, and cognitive reflection 
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